Action-Aware Embedding Enhancement for Image-Text Retrieval

Image-text retrieval plays a central role in bridging vision and language, which aims to reduce the semantic discrepancy between images texts. Most of existing works rely on refined words objects representation through data-oriented method capture word-object cooccurrence. Such approaches are prone ignore asymmetric action relation texts, that is, text has explicit (i.e., verb phrase) while image only contains implicit information. In this paper, we propose Action-aware Memory-Enhanced embedding (AME) for image-text retrieval, emphasize information when mapping texts into shared space. Specifically, integrate prediction along with an action-aware memory bank enrich features action-similar features. The effectiveness our proposed AME is verified by comprehensive experimental results two benchmark datasets.